Over the past decades, Augmented Reality (AR) technology has been applied in many application fields, such as medical and military training, engineering design and prototyping, tele-manipulation and tele-presence, and personal entertainment systems. See-through Head-Mounted Displays (ST-HMD) are one of the enabling technologies of an augmented reality system for merging virtual views with a physical scene. There are two types of ST-HMDs: optical and video (J. Rolland and H. Fuchs, “Optical versus video see-through head mounted. displays,” In Fundamentals of Wearable Computers and Augmented Reality, pp. 113-157, 2001.). The major drawbacks of the video see-through approach include: degradation of the image quality of the see-through view; image lag due to processing of the incoming video stream; potentially loss of the see-through view due to hardware/software malfunction. In contrast, the optical see-through HMD (OST-HMD) provides a direct view of the real world through a beamsplitter and thus has minimal affects to the view of the real world. It is highly (preferred in demanding applications where a user's awareness to the live environment is paramount.
Developing optical see-through HMDs, however, confronts complicated technical challenges. One of the critical issues lies in that the virtual views in an OST-HMD appear “ghost-like” and are floating in the real world due to the lack of the occlusion capability. FIG. 1 shows a comparison illustration of the augmented view seen through a typical OST-HWID (FIG. 1a) and the augmented view seen through an occlusion capable OST-HMD (OCOST-HMD) system (FIG. 1b), In the figure, a virtual car model is superimposed on a solid platform which represents a real object. Without proper occlusion management as shown in FIG. 1a, in a typical AR view, the car is mixed with the platform and it is difficult to distinguish the depth relationship of the car and the platform. On the contrary, with proper occlusion management as shown in FIG. 1b, the car blocks a portion of the platform and it can be clearly identified that the car seats on the top of the platform. Adding occlusion capability to the AR display enables realistically merging virtual objects into the real environment. Such occlusion-enabled capability may generate transformative impacts on AR display technology and is very appealing for many augmented-reality based applications.
An OCOST-HMD system typically comprises of two key sub-systems. The first is an eyepiece optics that allows a user to see a magnified image displayed on a microdisplay; and the second is a relay optics that collects and modulates the light from an external scene in the real world, which enables the opaqueness and occlusion control on the external scene when presenting to the viewers. The key challenges of creating truly portable and lightweight OCOST-HMD system lies in addressing three cornerstone issues: (1) an optical scheme that allows the integration of the two subsystems without adding significant weight and volume to the system. (2) a proper optical method that maintains the parity of the coordinate system of the external scene; (3) an optical design method that enables the design of these optical subsystems with an elegant form factor, which has been a persisting dream for HMD developers. Several occlusion-capable optical ST-HMD concepts have been developed (U.S. Pat. No. 7,639,208 B1, Kiyokawa, K., Kurata, Y., and Ohno, H., “An Optical See-through Display for Mutual Occlusion with a Real-time Stereo Vision System,” Elsevier Computer & Graphics, Special Issue on “Mixed Realities—Beyond. Conventions,” Vol, 25, No. 5, pp. 2765-779, 2001. K. Kiyokawa, M, Billinghurst, B. Campbell, E. Woods, “An Occlusion-Capable Optical See-through Head Mount Display for Supporting Co-located Collaboration,” ISMAR 2003, pp, 133-141). For example, Kiyokawa et. al. developed ELMO series occlusion displays using conventional lenses, prisms and minors. Not only because of the number of elements being used, but also more importantly due to the rotationally symmetric nature of the optical systems, the existing occlusion-capable OST-HMDs have a helmet-like, bulky form factor. They have been used exclusively in laboratory environments due to the heavy weight and cumbersome design. The cumbersome, helmet-like form factor prevents the acceptance of the technology for many demanding and emerging applications.